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1 The main result. 



The irreducible characters \ X °f the symmetric group & n are indexed by 
partitions A of n (denoted A h n or |A| = n), as discussed e.g. in 0, §1.7] or 
II §7.18]. If w G & n has cycle type v h n then we write x ( u ) f° r X X ( W )- If 
A has exactly p parts, all equal to q, then we say that A has rectangular shape 
and write A = p x q. In this paper we give a new formula for the values of 
the character x pxq - 

Let /i be a partition of k < n, and let (/i, l n_fc ) be the partition obtained 
by adding n — k l's to /i. Thus (/i, l n-fc ) h n. Define the normalized character 
X A GM n ~ fc ) by 



where x A (l n ) denotes the dimension of the character % A and (n)jt = n(n — 
1) • • • (n - k + 1). Thus |, (7.6)(ii)]H p. 349] x A (l n ) is the number f x of 
standard Young tableaux of shape A. Identify A with its diagram '■ 
1 < j < Aj}, and regard the points G A as squares (forming the Young 
diagram of A). We write diagrams in "English notation," with the first 
coordinate increasing from top to bottom and the second coordinate from 
left to right. Let A = (Ai,A2,---) and A' = (X^, X' 2 , . . .), where A' is the 
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conjugate partition to A. The hook length of the square u = E A is 

defined by 

h(u) — Xi + X'j — i — j + 1, 

and the Frame- Robinson-Thrall hook length formula |5], Exam. 1.5.2] [0, Cor. 7.21.6] 
states that 



For w e (5 n let denote the number of cycles of w (in the disjoint 

cycle decomposition of w). The main result of this paper is the following. 

Theorem 1. Let \i h k and fix a permutation E &k of cycle type \x. 
Then 

F X V, = (-i) fc E p Kiu) (-q) K(v \ 

UV=11>ij, 

where the sum ranges over all k\ pairs (u,v) E &k x ©fc satisfying uv = w^. 

The proof of Theorem 1 hinges on a combinatorial identity involving hook 
lengths and contents. Recall || Exam. 1.1.3] p, p. 373] that the content c(u) 
of the square u = (i, j) E A is defined by c(u) = j — i. We write s\(l p ) for the 
Schur function s\ evaluated at = for i > p. A well 

known identity [|5], Exam. 1.3.4] p, Cor. 7.21.4] in the theory of symmetric 
functions asserts that 

Since the right-hand side is a polynomial in p, it makes sense to define 

Equivalently, sa(1~ 9 ) = ( — 1)' A ' s a'(1 9 )- Regard p and g as fixed, and let 
A = (Ai, . . . , A p ) C p x q (containment of diagrams). Define the partition 
A= (Ai,...,A p ) by 

Xi = q — A p+ i_j. (3) 

Thus the diagram of A is obtained by removing from the bottom-right corner 
of p x q the diagram of A rotated 180°. Write 

#A=n^ 



«eA 



the product of the hook lengths of A. 

Lemma. With notation as above we have 

H pxq = (-lf\H x H~ x s x (l p )s x {r< 1 ). 

Proof. Let \^ denote the shape A rotated 180°. Let SQ(A) denote the 
skew shape obtained by removing from the lower right-hand corner of p x q 
and adjoining A^ at the right-hand end of the top edge of p x q and at the 
bottom end of the left edge. See Figure [| for the case p = 4, q = 6, and 
A = (4,3, 1). It follows immediately from P, Thm. 1] that 

#SQ(A) = H x H(p + c(u))H(q + c(u)) 

= (-l) lXl H~ x l[(p + c(u))(-q + c( U )). (4) 

It was proved in that the multiset of hook lengths of the shape SQ(A) 
is the union of those of the shapes p x q and A, so 

Hsq(x) = H pxq H X - (5) 
The proof now follows from equations ([!]), (H), (d), and (H). □ 

Proof of Theorem 1. Let i = We first obtain an expression for 

X pX9 (yU, l pq ~ k ) using the Murnaghan-Nakayama rule Exam. 1.7.5] || Thm. 
7.17.3]. According to this rule, 

T 

where T ranges over all border-strip tableaux (B 1 , B 2 , . . . , Bi +pq _ k ) of shape 
p x q and type (/x, l n_fc ) . Here we are regarding T as a sequence of border 
strips removed successively from the shape p x q. (See or |8j for further 
details.) The first i border strips B\, . . . , Be will occupy some shape A h k, 
rotated 180°, in the lower right-hand corner of p x q. If we fix this shape A, 
then the number of choices for B^, . . . , Be, weighted by (— l) ht (- Bl ) H \-ht(B e )^ j g 
by the Murnaghan-Nakayama rule just x A (aO- The remaining border strips 
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Figure 1: The shape SQ(4, 3, 1) for p = 4, q = 6 



Be + i, . . . , Bi +pq _k ah have one square (and hence height 0) and can be added 
in f x ways, where A has the same meaning as in (||). Hence 

XCpxq 
Xhk 



SO 



XCpxq 
Xhk 



(pq) k H pxq ^ a / \ 



(pq-k)\ 



fa) 1 ^ q H 'x 

Xhk 

H P , q £ xV)^ 1 - (6) 



ACpXq 
Ahfe 



Now let p(iy) denote the cycle type of a permutation w G The 
following identity appears in [^|, Prop. 2.2] and || Exer. 7.70]: 

^2H x s x ( y x)sx(y)sx(z) = — £ Pp(u)(x)Pp(v)(y)Pp(w)(z), 

X\-k ' uvw=l 

in <5 t, 
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where p v (x) is a power sum symmetric function in the variables x = (x\,X2, ■ ■ ■) 
Set x = l p , y = l~ q , take the scalar product (as defined in || §1.4] or || 
§7.9]) of both sides with p M , and multiply by (— l) fc . Since (in standard sym- 
metric function notation) the number of permutations in & k of cycle type [i 
is k\/z^ and since ip^p^) = z^ and (s A ,p M ) = x X (n), w e get 

(-l) k "£Hxsx(l p )sx(l- q )x X (») = (-1)" £ P Kiu \-<l) K{v) ■ (7) 

Ahfc uv=w^ 

Note that sa(1 p )sa(1~ 9 ) = unless A C p x g. Hence we can assume that 
A C p x g in the sum on the left-hand side of ([?[). 

Now the coefficient of x A (/ i ) i n @ is HpxqH^ 1 , while the coefficient of 
X A (aO on the left-hand side of (0) is (— l) fc iifxsa(l p )s,\(l~ 9 ). By the lemma 
these two coefficients are equal, and the proof follows. □ 



2 Generalizations. 



The next step after rectangular shapes would be shapes that are the union 
of two rectangles, then three rectangles, etc. Figure ^| shows a shape o h 
Y^LiPili that is a union of m rectangles of sizes Pi x q i: where q\ > q 2 > 
■■■ > q m - 

Proposition 1. Let o be the shape in Figure [J, and fix k > 1. Set 

n = \a\ and 

F k (j) 1 ,...,p m ;q 1 ,...,q m ) = x a (k,l n " k ). 

Then F k {pi, . . . ,p m ; q x , . . . , q m ) is a polynomial function of the Pi 's and qi 's 
with integer coefficients, satisfying 

(-l) fe F fc (l, . . . , 1; -1, . . . , -1) = (k + m - l) k . 

Proof. Let A = (Ai, . . . , A r ) hn and 

/i = (/ii, . . . , p, r ) = (Ai + r - 1, A 2 + r - 2, . . . , A r ). 

Define (p(x) = YYi=i( x ~ fa)- A theorem of Frobenius (see Exam. 1.7.7]) 
asserts that 

^<M-H>-]<^>, (8) 



p i 



q, 




Figure 2: A union of m rectangles 

where [x~ 1 ]f(x) denotes the coefficient of x~ x in the expansion of f(x) in 
descending powers of Taylor series at x = oo). 

If we let A = a in (|8|) and cancel common factors from the numerator and 
denominator, we obtain 



n—k\ 



say. Since 



{x)k Y\i X ~ (* + Pi + H 1" Pm))( 



— — \X 



1 



i=l 



|(x - (qi + p i+ i + p i+2 H h 



(9) 



1 



1 a a 
- + -77 + + 



x — a x x 2 x 3 



it is clear that [x ^H^x) will be a polynomial F k (pi, . . . ,p m ; q x , . . . , g m ) in 
the p,j's and g^'s with integer coefficients. If we put Pi = 1 and g« = —1 then 
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we obtain (after cancelling common factors) 

F t (l, . . . , 1; -1, . . . , -1) = _![,-.] («-*+lK«-m+D. . 

Since the sum of the residues of a rational function R(x) in the extended 
complex plane is 0, it follows that 

l r _ ln (x — k + l)(x — m + l) k 1 fix - k + l)(x - m + l)i 

\x = — — Kes T 



k x + 1 k \ x + 1 

= {-m) k 

= (-l) k (k + m-l) k . 

It remains to show that the coefficients of Fk(pi, . . . ,p m ; q±, . . . , q m ) are inte- 
gers. Equivalently, the coefficients of the polynomial 

r __n (x)k<p{x - k) 

are divisible by k. But 

(x) k ip(x-k) _ 

= {x) k (mod k) 

<p(x) 

and 

[x- 1 ](x) k = 0, 

so the proof follows. □ 

Note. For any fixed \i h k, J. Katriel has shown (private communica- 
tion), based on a method M for expressing x (ft, l n ~ k ) in terms of the values 
X A (j, l n_J ), that \ a (ft, l n_fc ) is a polynomial . . . ,p m ; gi, . . . , g m ) with 

rational coefficients satisfying 

(-1)^(1, ■ • • , 1; —1, ■ ■ ■ , —1) — (k + m- l) k . 

It can be deduced from the Murnaghan-Nakayama rule that in fact the func- 
tion F^pi, . . . ,p m ; qi, . . . , q m ) is a polynomial with integer coefficients. We 
conjecture that in fact the coefficients of ^(pi, . . . ,p m ; ?i, • • • , q m ) are non- 
negative: 
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Conjecture 1. For fixed \x h k, x a (fi, l n ~ k ) is a polynomial . . . ,p m ', 

qi, . . . , q m ) with integer coefficients such that (— l) fc F M (pi, . . . ,p m ; —qi, . . . , — q, 
has nonnegative coefficients summing to (k + m — l)k- 

We do not have a conjectured combinatorial interpretation of the coef- 
ficients of ( — l) k F^(pi, . . . ,p m ; — qi, . . . , — q m ). When m = 2 we have the 
following data, where we write a — pi, p — p 2 , b — q±, q = q2- 

-Fi(a,j?;-6,-g) = ab + pq 

F 2 (a, p; —b, —q) = a?b + ab 2 + 2apq + p 2 q + pq 2 
—F 3 (a,p;—b,—q) = a 3 b + 3a 2 b 2 + 3a 2 pq + ab 3 + 3abpq + 3ap 2 q + 3apq 2 

+p 3 q + 3p 2 q 2 + pq 3 + ab + pq 

F 4 (a, p; -b-q) = a% + 6a 3 b 2 + Aa 3 pq + Qa 2 b 3 + 12a 2 bpq + 6a 2 p 2 q 



We can say something more specific about the leading terms of Fk(p±, . . . ,p m ; 
qi, . . . , q m ). Let Gk(pi, ■ ■ ■ ,p m ] 9i> • • • j 1m) denote these leading terms, viz., 
the terms of total degree k + 1. 

Proposition 2. We have 



+6a 2 pq 2 + ab 4 + 4ab 2 pq + 4abp 2 q + Aabpq 2 + 4ap 3 q 
+14ap 2 q 2 + 4apq 3 + p 4 q + Qp 3 q 2 + 6p 2 q 3 + pq 4 + 5a 2 b 
+5ab 2 + lOapq + 5p 2 q + 5pq 2 . 




1 



(10) 



/ 



rn 



\ 



(-1) 



X 



- (qi + p i+1 + p i+ 2 H h p m )ar) 



i=l 
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Proof. From (Q) we have 

~[(x - (q { +Pi+ Pi+i H h p m )Y 



m 



G*(pi,..., J?*; ft., ...,&) = --[x 

k 



x 

1 r j=l 



^(X - (ft + p i+ i + H h p m ))' 



1=1 

say. Let L(l/x) = M(x)/x, so M(0) = 1. Regard M(x) as a power series in 
ascending powers of x, i.e., an ordinary Taylor series at x = 0. Then by the 
Lagrange inversion formula || Thm. 5.4.2] we have 

[x" 1 ]L(x) fc = [x fc+1 ]M(x) fc = -&[x fc ]- 



[x/M{x)Y- 1 )' 
so equation (TO) follows. □ 

Proposition 2 was also proved by Philippe Biane (private communication) 
in the same way as here, though using the language of free probability theory. 

It follows from Proposition 1 or Proposition 2 that (— l) h Gk{pi, ■ ■ ■ ,p m ] 
—qi, . . . , —q m ) is a polynomial with integer coefficients summing to 

^:=(-l) fc G fc (l,...,l;-l,...,-l). 

From Proposition 2 we have 



~ ( x(l-x) V" 1 ' 



— (m — l)x 

an algebraic function of degree two. When m = 1 we have = C^, the 
fcth Catalan number. Hence by Theorem 1 Ck is equal to the number of 
pairs (u, v) G &k x ©fc such that k(u) + = fc + 1 and uv = (1,2, ... ,k), 
a known result (e.g., || Exer. 6.19(hh)]). Moreover, it follows easily from 
Proposition 2 that 

k 

(-l) k G k (p; -q) = Y,N(k,i)p k+l - l q\ 
i=i 



where N(k,i) = |( fe ) (•*]), a Narayana number ||, Exer 6.36]. Hence N(k,i) 
is equal to the number of pairs (u, v) G x such that «;(«) = z, = 
fc + 1 — i, and m; = (1, 2, . . . , k). When m = 2 we have = r k , a (big) 
Schroder number || p. 178]. 

It would follow from Conjecture 1 that the polynomial (— l) k Gk{pi, ■ ■ ■ ,p m ', 
— qi, . . . , — q m ) has nonnegative coefficients. In fact, Sergi Elizalde has shown 
(private communication of May, 2002) that 

(-l)*G fc (pi, . . . ,p m ; -q u . . . , -q m ) 

-I E ©(G)) 

n(Te)(( i -0)C" r " <, "'"^." A """"^)) 

i F m yi y m > 

where (£) = ( <i+ ^" 1 )- Thus in particular {-l) k G k {p u ...,p m ] -qi, -q m ) 
indeed does have nonnegative coefficients. Do they have a simple combina- 
torial interpretation? 
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